The following report contains model performance metrics for the NY City Hourly Probabilistic Residential Energy Demand Forecasting Pipeline.
Model performance was evaluated on both long-term and day-ahead forecasts. Evaluation was conducted using a holdout dataset of hourly energy
demand values between 2023-09-26 and 2024-04-27.
Long-Term Forecasting Performance
The following table contains performance metrics for the forecasting model compared with a yearly moving average baseline model.
|
MSE |
Weighted MSE |
MAE |
MAPE |
Wilcoxon Test p-value |
| Forecasting Pipeline |
128742.60 |
126301.35 |
270.23 |
0.05 |
0.0 |
| Baseline Yearly MA |
604819.81 |
604819.81 |
626.46 |
0.13 |
NaN |
The following Plotly Figure helps to contextualize the forecasting model's performance by showing its predictions along with the actual energy demand
values. It also presents the 95% confidence interval bounds estimated by the forecasting model.
Day-Ahead Forecasting Performance
The following table contains performance metrics for the forecasting model compared with a yearly moving average baseline model.
|
MSE |
Weighted MSE |
MAE |
MAPE |
Wilcoxon Test p-value |
| Forecasting Pipeline |
125951.15 |
125950.90 |
271.18 |
0.05 |
NaN |
| Baseline Moving Avg |
604819.81 |
604819.81 |
626.46 |
0.13 |
0.0 |
| EIA Forecasts |
113145.06 |
NaN |
291.30 |
0.06 |
0.0 |
The following Plotly Figure helps to contextualize the forecasting model's performance by showing its predictions along with the actual energy demand
values. It also presents the 95% confidence interval bounds estimated by the forecasting model.
The following data has been made up for demonstration purposes. Actual data will be provided in a future update.